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About the National Reading Research Centei 



The National Reading Research Center (NRRC) is 
funded by the Office of Educational Research and 
Improvement of the U.S. Department of Education to 
conduct research on reading and reading instruction. 
The NRRC is operated by a consortium of the University 
of Georgia and the University of Maryland College Park 
in collaboration with researchers at several institutions 
nationwide. 

The NRRC’s mission is to discover and document 
those conditions in homes, schools, and communities 
that encourage children to become skilled, enthusiastic, 
lifelong readers. NRRC researchers are committed to 
advancing the development of instructional programs 
jensitivetothecognitive, sociocultural, and motivational 
factors that affect children’s success in reading. NRRC 
•researchers from a variety of disciplines conduct studies 
with teachers and students from widely diverse cultural 
and socioeconomic backgrounds in pre-kindergarten 
through grade 12 classrooms. Research projects deal 
with the influence of family and family-school interac- 
tions on the development of literacy; the interaction of 
sociocultural factors and motivation to read; the impact 
of literature-based reading programs on reading achieve- 
ment; the effects of reading strategies instruction on 
comprehension and critical thinking in literature, sci- 
ence, and history; the influence of innovative group 
participation structures on motivation and learning; the 
potential of computer technology to enhance literacy; 
and the development of methods and standards for 
alternative literacy assessments. 

The NRRC is further committed to the participation 
of teachers as full partners in its research. A better un- 
derstanding of how teachers view the development of 



literacy, how they use knowledge from research, and 
how they approach change in the classroom is crucial to 
improving instruction. To further this understanding, the 
NRRC conducts school-based research in which teach- 
ers explore their own philosophical and pedagogical 
orientations and trace their professional growth. 

Dissemination is an important feature of NRRC 
activities. Information on NRRC research appears in 
several formats. Research Reports communicate the 
results of original research or synthesize the findings of 
several lines of inquiry. They are written primarily for 
researchers studying various areas of reading and 
reading instruction. The Perspective Series presents a 
wide range of publications, from calls for research and 
commentary on research and practice to first-person 
accounts of experiences in schools. Instructional Re- 
sources include curriculum materials, instructional 
guides, and materials for professional growth, designed 
primarily for teachers. 

For more infprmation about the NRRC’s research 
projects and other activities, or to have your name 
added to the mailing list, please contact: 

Donna E. Alvermann, Co-Director 
National Reading Research Center 
318 Aderhold Hall 
University of Georgia 
Athens, GA 30602-7125 
(706) 542-3674 

John T. Guthrie, Co-Director 
National Reading Research Center 
3216 J. M. Patterson Building 
University of Maryland 
College Park, MD 20742 
(301)405-8035 



7 




ERIC 



s 



t 



I 



Peter P. Afllerbach is an Associate Professor in the Depart- 
ment of Curriculum and Instruction and a Principal Investigator 
at the National Reading Research Center at the University of 
Maryland College Park. Dr. Afflerbach’s research interests 
include reading assessment that is conducted at national, state, 
and district levels, as well as reading assessment that is conduct- 
ed by teachers in classrooms. He is the editor of a recent book 
on statewide reading assessment initiatives ( Issues in statewide 
reading assessment ), and the co-editor of a recent book that 
examines innovation in reading assessment at the classroom, 
district, and state levels ( Authentic assessment: Practices and 
possibilities). Dr. Afflerbach is also interested in the cognitive 
processes involved in reading and recently co-authored the book 
Verbal protocols of reading : The nature of constructively 
responsive reading. He also co-edited the forthcoming volume 
Developing engaged readers in home and school communities , 
which represents the initial work of the National Reading 
Research Center. His most recent research focuses on how 
reading assessment information is communicated, the barriers 
to the implementation of statewide performance assessments, 
&id the content validity of the 1994 NAEP reading framework. 
u Dr. Afflerbach has been a member of the Literacy Assess- 
ment Committee of the International Reading Association, and 
he currently serves on the Reading/Language Arts Advisory 
Panel of the New Standards Projects, the Content Area Experts 
Panel for Middle School for the National Board of Professional 
Teaching Standards, and on the Board of Directors of the 
National Reading Conference. He was selected as a Lilly 
Foundation Fellow for the Excellence in Teaching Program at 
the University of Maryland, and was awarded a Ford Founda- 
tion Grant for innovation in undergraduate education when he 
taught at Emory University. 

Karen Moni received an Honour Degree in English literature 
and the Diploma of Education at the University of Leeds, 
England. She then served as an English and drama teacher in an 
English high school. After moving to Australia she taught in an 
alternative education program for disaffected youth and school 
refusers in Brisbane, Queensland. 

Karen Moni spent two years at the National Reading 
Research Center at the University of Maryland before returning 
to Australia. She is currently pursuing a doctoral degree in 
literacy assessment while serving as a research assistant at the 
Schonell Special Education Research Center at the University 
of Queensland, Brisbane,. Australia. 





National Reading Research Center 
Universities of Georgia and Maryland 
Instructional Resource No. 33 
Fall 1996 



Improving the Usefulness and 
Effectiveness of Reading Assessment 



Abstract. There is considerable interest in reading assess- 
ment and the influence that reading assessment can have on 
what is taught and learned in school. However, little is known 
about how teachers, administrators, parents, and students 
communicate and understand reading assessment informa- 
tion. Lacking this knowledge, we do not know what kinds of 
reading assessment are useful to teacher and students. We 
cannot determine how (and what kind of) reading assessment 
contributes to effective reading instruction and learning. The 
study we draw on in this instructional resource investigated 
the communication of reading assessment information be- 
tween teachers and students in a metropolitan elementary 
school. We focus bn the reading assessment information that 
we encountered in a sixth-grade classroom, and we describe 
the experiences and issues that emerged in our yearlong 
investigation of how the teacher and a student communicated 
and understood reading assessment information. Following 
the description of communications related to each type of 
reading assessment, we provide recommendations for best 
utilizing reading assessment information. 

Reading assessment is central to the maintenance 
and improvement of educational practice and learning 
(Valencia, Hiebert, & Afflerbach, 1994). Assessment 
results provide diagnostic information that may lead to 
changes in instruction and learning in a classroom, 
school, district, state, or country (Johnston, 1992). 
Assessment is also used to hold students, teachers, 



parents, and schools accountable, and assessment can 
measure students’ minimum competencies as well as 
their maximum achievements. Reading assessment 
information may prove motivating for teachers and 
students as it provides feedback about the quality and 
accuracy of reading performance. In addition, reading 
assessment results may influence school funding 
decisions, and the endorsement or rejection of particu- 
lar instructional programs. Despite much recent interest 
and research in reading assessment and the consider- 
able influence that reading assessment can have on 
what is taught and learned in school, little is known 
about how teachers, administrators, parents, and 
students communicate andunderstand reading assess- 
meiaJ information. Without this knowledge we do not 
know what kinds of reading assessment are useful to 
teachers and students. Furthermore, we are unable to 
determine how, (and what kind of) reading assessment 
contributes to effective reading instruction and learning, 
and we are unable to judge the ultimate effect or 
influence of assessment on teaching, learning, and 
achievement. 

This study investigated the communication of 
reading assessment information between teachers and 
students in a metropolitan elementary school. We 
examined the intended and interpreted meaning of 
reading assessment information including teachers’ 
oral feedback to students during reading lessons, 
report cards, and a district-wide criterion referenced 
test. In this instructional resource, we will focus on the 
reading assessment information that we encountered 
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in the sixth-grade classroom of Karol; and we describe 
the manner in which William, a student in Karol’s class, 
understood the information. We will describe the 
experiences and issues that emerged in our yearlong 
investigation of how Karol and William communicated 
and understood reading assessment information. 
Following the examination of each type of reading 
assessment, we will provide recommendations for 
school personnel that might help best utilize the infor- 
mation from specific assessment materials and practices, 
and that might inform the development and selection of 
reading assessments in elementary school. 

Methodology 

This study was conducted over the course of an 
academic year, from September to June. We used 
interpretive, phenomenological inquiry(Erickson, 1986) 
to construct an understanding of how people communi- 
cate reading assessment information, and how they 
assign meaning to both reading assessment informa- 
tion .and reading assessment processes (Bogdan & 
Biklan, 1982; Guba & Lincoln, 1989). 

The study was conducted in a public school in a 
large metropolitan district. Fern Hill Elementary School 
is situated in an ethnically diverse community, and the 
school population is comprised of students labeled by 
the district as Black (66%), Hispanic (24%), White 
(5%), and Asian (4%). Fern Hill Elementary School is 
located in one of the 20 largest school districts in the 
United States. Both the school and the district are 
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struggling to meet the standards in reading and writing 
that were established as part of a statewide school 
performance program. 

Data Collection and Analysis 

We used an array of materials and procedures to 
gather data to describe the communication and inter- 
pretation of reading assessment information. The data 
gathering instruments and procedures included inter- 
views; classroom observations; videotapes of reading 
lessons; think-aloud protocols of the construction and 
interpretation of assessment information; examination 
of heading assessment materials and related docu- 
ments at the classroom, school, and state levels; and 
ratihg’tasks. 

Participants \ 

Ourexamination ofhow Karol and William communi- 
cate and understand reading assessment is taken from 
a larger study that included two teachers and four of 
their students. Karol has taught sixth grade for 4% 
years at Fern Hill. Her training was originally in art 
education and her graduate courses in the master’s 
program did not include reading or reading assessment 
courses. She recently completed a district-sponsored 
workshop in classroom-based assessment. Karol is a 
reflective teacher who told us that she is constantly 
searching for effective ways to help her students 
achieve their goals. She also told us she is interested 
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in reading assessment and regularly seeks the most 
effective methods for assessing students. She has high 
expectations that students should care about their work 
and their achievements. Karol told us that she had the 
goals of helping students create respect for them- 
selves, their teacher, and others. Karol wants each of 
her students to achieve to their ability, and she treats 
each student within this frame of expectation that they 
should do their best. 

Karol believes that the most valuable reading 
assessment occurs within the daily classroom routines 
of oral reading and teacher feedback. Question and 
answer sessions and listening to student discussions 
tells her most about how the students are doing. She 
bdtieves that the daily contact and interaction that she 
has with her students tells her more about their prog- 
refcslhan the district-wide and statewide assessments. 
Students in Karol’s class regularly use work sheets and 
the worksheet grades represent both accountability to 
school district grading requirements and evidence that 
supports her classroom observations. Karol tells us that 
the grades “are important to me as far as having 
something concrete that I have to go back to ... and to 
give grades in a book.” While her major concern for her 
students is reading comprehension, her major goals 
are to get the students “psyched up about reading,” 
and for students to learn “a desire and a love of read- 
ing.” Karol uses the basal reader series because, “I like 
it ... there’s a nice theme builder ... and the selections 
in each unit are good stories.” She also tries to give her 
students choices in their reading through the use of 



0 

ERIC 



5 



13 



additional literature. 

Prior to the beginning of the school year we asked 
Karol to recommend students who might be willing to 
participate in the study. Karol enlisted the help of 
William, whom Karol considered to be a developing 
reader. William is a highly motivated African-American 
student who makes great effort to succeed in school 
and to please his teacher. William participates con- 
stantly: His hand is held high up in class when Karol is 
calling on students to read or to respond to questions. 
William is persistent even though he is sometimes “just 
way off base with his thoughts,” according to Karol. 
While William is an enthusiastic and often motivated 
reader, he is not one of the stronger readers in his 
clas£. William considers reading to be “very important." 
Wh^n we asked him why, he told us: 

You have to understand the skill ... like SQR ... 
which is like,... survey, question, read, recite, and 
review ... aftd you read ... like I want to be a 
lawyer ... so I have to understand ... this ... what 
is it about .... what is the case about? 

When we first asked William how he knew he was a 
good reader, he told us that he considers himself a 
good reader because Karol tells him so. 

When Karol says “You’re a good reader ... very 
good”... but I think I can understand the skills 
more than I can read. 

William occasionally used self-assessment, and he told 
us that he knew he needed to work to improve his 
reading, especially sounding out words. 

Investigator: What do you need to work on? 
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William: Some of my words 

Investigator: How do you know? 

William: Because when I read I stutter ... and I 

can’t pronounce the words right. 

William believes he is doing “all right” in reading this 
year. He also tells us that Karol knows how he is doing 
in reading by paying attention to “the way I read ... by 
the way I put accent on the words ... and reading out 
loud." In addition to Karol’s verbal feedback, William 
reports that he knows how well he is doing “because of 
my report card and the grades I get for my work.” 
William also reports that his performance on activities 
and tasks helps him understand himself as a reader. 
Outlining ...comprehension skills ... story map ... 
and summary.... you have to know the story to 
-flo the work ... so to do it you know you’re doing 
a good job. 

William achieved the report card grade B in reading 
across the year (a consistent 3.0 GPA). He believes 
that the “B” grade means “you’re not excellent, but you 
know what you’re doing.” William considers teacher- 
determined grades and Karol’s oral feedback as the 
most important communication about his development 
as a reader. William demonstrates a habit of depending 
on reading assessment information given by Karol to 
gauge his ability as a reader, across the school year. 
He sometimes augments this with reflection on his own 
work. 

Teacher Oral Feedback During Reading Lessons 

Our first investigation of the communication and 
interpretation of reading assessment information 
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examined Karol’s oral feedback to William during class 
reading lessons. We determined through interviews 
and observations that this was the most valued and 
frequent form of reading assessment information 
encountered in Karol’s classroom (Afflerbach & Moni, 
1995). We also determined that classroom discourse 
mostfrequentlytookthelRE(initiate-respond-evaluate) 
form (Cazden, 1978; Mehan, 1978). A coding scheme 
was developed which allowed us to determine the 
relationship between Karol’s intended meaning and 
what William understood from Karol’s feedback. The 
coding scheme included the categories of congruent, 
partially congruent, and not congruent. 

We videotaped classroom reading lessons, and 
askgd Karol to observe the videotape and describe the 
intended meaning of herfeedback. William then viewed 
the yideotape and told us his understanding of Karol’s 
feedback. The following excerpt is from a lesson in 
which William worked with a basal reader worksheet. 
Students read Statements and Karol calls upon individ- 
ual students to decide whether statements are facts or 
opinions. In the excerpt, William does not understand 
the focus of the message (a particular word) or the 
related comment of Karol’s message (to inform William 
that the answer is incorrect). 

Example of incongruence in intended 
and interpreted meaning of teacher feedback 
to student response 

Classroom interaction 

K: We’re talking about doctors perform amazing 
operations 
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W: I think that’s ... opinion 
K: Why? 

W: Because they could have just like minor.... 

K: There’s one word in there that really makes it an 
opinion 

Karol's intention 

K: I didn’t make a comment to him because he was 
off... I didn’t say— “no that’s wrong” ... I just went to 
giving them there’s one word that’s going to tell 
you that it’s an opinion ... he didn’t get the word ... 
I didn’t want to tell that in front of the whole class 
... I just wanted to to give him another hint 
William’s interpretation 

I: How did you do there in answering the question? 

W: Good because she said.... I don’t really know if she 
meant the sentence I said ... but I think I did a 
tjood job. 

(BI2/8:23-9:00) 

William’s inability to construct meaning that was 
congruent with Karol’s intended message was noted in 
nearly half the feedback sequences we analyzed in this 
study; although, William was sometimes able to con- 
struct congruent and detailed meanings from seemingly 
simple teacher feedback, such as “OK,” or the repeti- 
tion of a student’s response. His ability to assign appro- 
priate meanings to this feedback appeared to be 
influenced by his familiarity with the classroom context, 
what he chose to attend to within the context, and the 
perceived roles and purposes of reading assessment. 

Researchers have demonstrated that the IRE 
pattern limits the nature of classroom discourse (Mehan, 
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1978), but the familiarity of the pattern may have 
helped William understand the assessment messages 
sent by Karol. In this case, we believe the predictable 
nature of spoken discourse in Karol’s classroom 
sometimes serves to support William’s understanding 
of teacher feedback: William’s experiences in Karol's 
classroom familiarize him with both the nature and 
purpose of teacher talk. The results indicate the impor- 
tance of familiarity with the culture and language of the 
classroom as a means of helping students understand 
the evaluation that is communicated by a teacher. 
Karol did not report teaching students howto construct 
congruent meanings from her evaluative feedback, nor 
did the students report receiving such training. Yet, 
William did understand Karol’s intended message in 
over.half the evaluative sequences that we analyzed. 
While we make no endorsement of the IRE pattern, we 
note that it was familiar to students and that it appeared 
to facilitate their understanding of teacher feedback. In 
contrast, Karol and her students might discuss the 
nature of classroom discourse (helping some students 
become metacognitive about the discourse), and this 
could contribute to increased understanding of the 
subtleties of the discourse, and their intent. 

Based on our findings, we have several recommen- 
dations that might help teachers help their students 
make the most of oral feedback during reading lessons. 
Classroom discussions of the meaning and purpose of 
feedback might help students best understand and use 
feedback that is offered by teachers during the daily 
reading lesson. Consistency in classroom discussion 
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format can help students develop familiarity with the 
purpose and meaning of their teacher’s feedback. The 
use of classroom charts and reminders that help 
students query the nature of the feedback they get from 
the teacher could prove valuable here. For example, 
charts to list, give examples of, and remind students of 
the types of teacher feedback and their meanings. 
Posters could also serve to remind students to pay 
close attention to teacher feedback and to seek clarifi- 
cation if it is not understood. Similarly, this might be 
accomplished through regular discussions with stu- 
dents about the purposes of teacher feedback, the 
important information to attend to in such information. 
Where the teacher and students are attempting to 
establish nontraditional discussion patterns, clarifying 
the purpose of feedback from teacher and peers may 
be dfchieved with detailed guidelines for giving and 
understanding oral feedback. 

We'believe it is of particular importance that teach- 
ers consider thejmplications that changing and nontra- 
ditional discussion patterns and formats have for 
student understanding of assessment feedback. For 
example, we know little about how students understand 
the evaluative feedback of peers. If there is movement 
in a classroom or school to replace or augment teacher- 
dominated reading lessons and the IRE discourse 
pattern with peer discussions, it is crucial that we help 
students become adept at communicating and inter- 
preting the reading feedback they exchange with their 
classmates. In summary, we believe that when stu- 
dents have clear expectations for the nature and 
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meaning of teachers’ oral feedback, they will be best 
situated to understand and make use of it. We found 
that the predictability of the classroom discourse 
appeared to make teacher feedback comprehensible. 

The Report Card 

The report card is an important means of communi- 
cating reading assessment information in Karol’s 
classroom. Early in this study, the assistant principal 
Barbara reported her specific concerns about misinter- 
pretations of reading assessmentinformation contained 
in report cards. She reported that experiences in her 
role as vice principal and testing coordinator height- 
ened her concerns about how students and parents 
understood the report card. 

The report card is as important to the student as 
to the parent, but the way it’s [the report card] set 
up is sometimes ... the wording is misleading ... 
we’ll have places for the grade ... the letter ... 
and then it’ll say “Writing not legible” and they 
won’t understand why they’re [the boxes] 
checked or not checked or left blank, and whether 
or not there is something in the comments 
section. 

On a Likert- scale task, William rated report card grades 
as “very important” reading assessment information 
that was “very easy” for him to understand. William also 
indicated that he believed that reading report card 
grades were very important to his parents, and to 
Karol. To understand how Karol constructed report 
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cards for her students, we asked her to think-aloud 
while writing report cards (Afflerbach & Johnston, 
1993). Karol provided the following think-aloud account 
of how she arrived at William’s reading grade: 

William really participates a lot ... much more 
than [student] does ... and he’s not always on 
target ... he’s more often ... especially on com- 
prehension questions ... I don’t think he really 
knows what he’s reading all the time ... he really 
needs work on slowing down and not being so 
concerned with the final grade as to really re- 
read things a couple of times through so that he 
understands what he’s reading ... I didn’t feel that 
he should get an A just because of his participa- 
tion ... because most of his grades are Cs and 
J3s ... so that’s how I came up with his grade ... 

C; 6, 6, C+, 6, C+ ... turned in every single 
assignment ... he didn’t give me a March book 
report ... and had I seen that last night or yester- 
day ... I really give them to the last possible ... 
the end of the grading period was last Wednes- 
day ... but they were still turning in work Monday 
and Tuesday ... that’s why I’m writing their report 
cards because I try to get everything graded ... I 
try to give them every opportunity to get work in 
... he’s still averaged out to a 2.54 GPA, but I 
really pushed those book reports; and I tell them 
if they don’t turn in their book reports, I bring 
them down a grade ... and I didn’t in his case.... 
William received a 6 for all three areas of reading 
(Reads with understanding, Uses word attack skills, 
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Shows wide interest in reading) and Karol recorded the 
following in the small written comments section: 
William’s work habits and consistent participation 
continue to be his strong points. I’m very happy 
to see the improvement in Language and Social 
Studies. He continues to shine in Math. I’m very 
disappointed in behavior and hope it improves in 
the 4th quarter. GPA 2.54. 

William provided the following interpretation of the 
grades and comments that he received from Karol: 

I: Tell me what your report card means to you. 

W: That I’m above average.... 

I: Can you tell me what above average means? 

W: It means that you’re not excellent ... you’re ... you 
‘‘know what you are doing. 

I: WVhat does your grade tell you? 

W: 'i know what I’m doing. 

I: Where do you think this grade comes from? 

W: From my bdok reports and my work sheets and the 
other work that we do in reading ... and the way I 
pronounce words ... and when I do my writing 
prompt ... and test scores from reading on the 
computer. 

In the above excerpt, William is able to describe 
some of the sources of his grade in detail, including 
book reports, worksheets, test scores, writing on the 
computer, and his pronunciation of words. He also uses 
the report card grading key to literally interpret his 
grade of 6. It tells him his work is "above average” and 
he does not reflect on his performance to provide 
details of this grade. He reports that the report card 
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grades tell him that “I know what I’m doing.” The report 
card appears to fulfill one of Karol’s goals (to affirm 
William’s contributions and regular participation), but 
there is a lack of communication of the concern about 
William’s development that is present in Karol’s think- 
aloud account. 

Geared to Karol’s goals, the report card served as 
affirmation and motivator for William. He interpreted the 
letter grades as representing general levels of achieve- 
ment. William was somewhat able to deconstruct his 
reading report card grades when prompted, but he 
exhibited only a general knowledge of the nature of 
information that Karol used in the report card grade 
construction task. 

Bgsed on our examination of how Karol constructed 
and communicated reading assessment information in 
the report card, and how William interpreted this 
information, we have several observations and recom- 
mendations. First, it is imperative that the report card is 
useful to teachers, students, and parents. This useful- 
ness revolves around the nature of the information con- 
tained in the report card and how well it can be inter- 
preted by readers of the report card. Usefulness of 
report card information appears related to teachers’ 
and students’ familiarity with structure and function of 
the report card. Karol did not know when the report 
card was created, nor did she know who created it. 
Furthermore, the report card did not appear to be 
aligned with Karol’s curricular goals. 

We recommend that school districts, supervisors, 
teachers, parents, and students be considered as 
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contributors to the report card development process. 
Through the consideration of the different audiences 
and their potential contributions, the greatest use of the 
report card may be realized. We also recommend that 
there be initial and periodic checks to determine how 
well the report card and the reading curriculum are 
aligned. Many report cards represent a behavioristic 
and decontextualized view of reading development, 
and this may not be an appropriate format for reporting 
progress in a reading program that embodies construc- 
tivist and literature- centered approaches to reading 
instruction. Our findings indicate that time should be 
alloted for teachers to explain reading report cards to 
students. During such time, teachers may clarify for 
sti&ients how a grade was determined and what stu- 
dent performances contributed to a high or low grade. 
Wf note that to the degree that a report card requires 
teacher narratives of student learning, there may be 
less need for; teacher/student meetings to interpret 
report cards. \A le note also that either approach to the 
communication and interpretation of reading assess- 
ment contained in the report card will be enhanced 
when sufficient time is provided in the school schedule 
to construct and explain grades. 

District-Wide Criterion Referenced Tests 

William understood the high stakes nature of the 
criterion-referenced tests (CRT). He also was familiar 
with the practice of practicing to take the test. He had 
a hazy notion of the connection of the district test to the 
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statewide performance assessment, and he knew that 
a report of his performance on the CRT (which con- 
sumed considerable school resources and time) was 
delayed: no score was reported back to his teacher for 
the entire year. The actual CRT administration lasts for 
1 week in early May. The spring testing provides the 
measures of student reading that are used by the 
county. Students at Fern Hill also take a practice CRT 
in January to help prepare for the spring testing. This 
orientation took place over the course of 1 week for 1 V 2 
hr per day (7!4 hr total). Students took the test again in 
May, again for 1 week. 

Despite the considerable time and effort invested in 
practicing for, administering, and taking the test, a 
district supervisor reported that information about 
individual students was not reported back to teachers 
or students by the end of the school year. 

Last year, there was the spring administration of 
the CRT ... and that information would not come 
back to the teacher with which the students took 
the test. 1 

Although there was increased effort to reduce the time 
between administering the test and reporting back of 
scores, the supervisor noted that this was possible only 
with the multiple choice items. The performance 
assessment items were not machine-scoreable, and 
the reporting time for these was considerable. 

We hope to have a very quick turn-around on the 
multiple choice ... within 3 to 4 weeks ... and the 
performance items ... the logistics of scoring 
them become a big concern ... the audience 
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would be teachers, building administrators, and 
' parents because parents would get back individ- 
ual information on how well their children did. 
William rated the assessment information yielded by 
the CRT as very important, and he believed that the 
information from the CRT “helps a lot” in him becoming 
a better reader. He did not tell us why or how the test 
did this. William rated the CRT as easy to understand 
and as very important to his parents and his teacher. 
William also reported that the CRT required “a lot of 
time” for the class to take and complete. When asked 
about the CRT, William gave the following answers 
I: Do you know the name of the test you’re taking? 

W: The CRT. 

I: Do you know who made the test? 

W: No. 

I: w Do you know what the test is for? 

W : | No. 

/; ■ Do you know if you’ll find out how you did on the 
test? • 

W: Well, I did find out in third grade. 

I: When do you think that will be? 

W: I’m not sure that I’ll ever know. 

I: What do you think the test scores will be used for? 

W: Like, what kind of reading group I’m in at ... not 
reading group but what kind of reading skills I’ll get 
next year. 

I: Is this test important for you? 

W: Yeah. 

I: Why? 

W: Like it’s something where you find out next year . . . 
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what kind of reading you’ll be in ... and I think 
nobody would want to be below average. 

Large-scale tests like the CRT presented a daunting 
set of challenges to the teachers and students at Fern 
Hill Elementary School. Developed outside of school, 
they were not accompanied by clear or helpful explana- 
tions of the purpose or uses of the test. It was difficult 
for teachers to communicate to students information 
related to different aspects of the tests. The teachers 
were also caught in a Catch-22 situation. The criterion- 
referenced test demanded considerable time for test 
preparation and administration. Because the teachers 
and students were sometimes uninformed partners in 
the criterionreferenced enterprise, teachers had to 
decitte if they would take even more time from their 
already constrained professional schedules to find out 
about the test, and to educate their students about the 
nature and purpipse of the tests. 

Based on Karol’s and William’s experiences with the 
district-wide criterion referenced test, we have several 
recommendations related to the development and use 
of large-scale reading assessments. First, efforts must 
be made to help teachers and students understand the 
purpose of the assessment and the uses of the assess- 
ment information. We found that Karol assumed a 
“middleperson” role in terms of the CRT. She under- 
stood some of the details of CRT purpose and use, but 
there was much that was not clear. To more fully 
understand the details of the CRT, Karol would have to 
invest further time to determine sources of information. 
Where Karol might find this time is not clear: the CRT 



preparation and administration already demanded a 
considerable amount of the school year. Clearly, efforts 
to develop new large-scale assessments must be 
matched by efforts to clearly describe the ways and 
means of the assessment. 

William’s experience with the CRT also brings us to 
several recommendations. First, students need to know 
why they are being assessed and what the assessment 
information will be used for. Lack of clarity in these 
issues may send students a subtle (or not so subtle) 
message that reading assessment is not their concern. 
Over time, this may create an expectation or belief 
among students that reading assessment is not some- 
thing they do or need to do. Second, we believe 
students will benefit from the feedback and closure that 
can be provided by the results of large-scale assess- 
ment. William did not receive any information about his 
performance on an assessment that demanded over 2 
weeks of reading class time. 

1 ' Conclusions 

In summaiy, the administrators, teachers, and 
students at Fern Hill Elementary School use a reading 
assessment system that has different components that 
serve different purposes. Teachers and students 
ascribe different meanings and purposes to these 
assessments, and the assessment materials are used 
differently as determined by perceived roles and contexts. 
What lessons did we learn from our collaboration with 
Karol and William at Fern Hill Elementary School? 



We learned that successful communication of 
reading assessment information is never guaranteed, 
and that careful work must be done to create a context 
in which students understand what is intended. Stu- 
dents and teachers should discuss reading assessment 
materials, procedures, and purposes as conducted in 
the classroom on a daily basis. This will familiarize 
them with the intent of assessment information, and 
may help them determine when communication is not 
working well. For reading assessment that is tied to 
initiatives and procedures developed outside the 
classroom (such as report cards and district or state 
tests), our advice is that teachers who have the oppor- 
tunity to serve on assessment development teams 
should seriously consider the contributions they might 
make to an effective and effectively communicated 
assessment program. School districts that are develop- 
ing assessments should take seriously the charge of 
developing a means of communicating information 
about the nature, means, and purposes of assessment 
as they are developing the assessment itself. Too 
often, there is little or no work done in this critical area. 

Finally, we believe it is critical to consider a priori 
the effects, uses, and misuses of any assessment 
materials and procedures that are considered for 
implementation in classrooms. The influences on 
teachers and students, the time it takes from the school 
day, and the time taken from other things which actu- 
ally might contribute to literacy all demand that reading 
assessment be examined so that the people who 
receive reading assessment information construct 
understandings that were intended by the senders. 
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